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Abstract 

Given  a  sequence  of  fractional  frequency  deviates,  we  investigate  the  relationship  between  the 
sample  variance  of  these  deviates  and  the  total  variance  (Totvar)  estimator  of  the  Allan  variance. 
We  demonstrate  that  we  can  recover  exactly  twice  the  sample  variance  by  renormalizing  the  Totvar 
estimator  and  then  summing  U  over  dyadic  averag^g  times  1,  2,  4,  2-'  along  with  one  additional 

term  that  represents  variations  at  all  dyadic  averaging  times  greater  than  2^.  This  decomposition  of 
the  sample  variance  mimics  a  similar  theoretical  decomposition  in  which  summing  the  true  Allan 
variance  over  all  possible  dyadic  averaging  times  yields  twice  the  process  variance.  We  also  establish 
a  relationship  between  the  Totvar  estimator  of  the  Allan  variance  and  a  biased  maximal  overlap 
estimator  that  uses  a  circularized  version  of  the  original  fractional  frequency  deviates. 


1  INTRODUCTION 

The  goal  of  this  paper  is  to  explore  the  relationship  between  the  sample  variance  of  a  sequence 
of  fractional  frequency  deviates  {yn  :n  =  namely, 

^y  =  iv  E (S'" - »)" > 

^  n=l  ^  n=1 

and  a  new  estimator  of  the  Allan  variance  called  “Totvar”  (“total  variance”  -  see  the  companion 
article  by  Howe  and  Greenhall  [1]  in  these  Proceedings  for  additional  details).  The  Totvar 
estimator  is  based  upon  the  hypothesis  that  reasonable  surrogates  for  unobserved  deviates  y„, 
n  <  1  or  n  >  iV„,  can  be  formed  by  tacking  on  reversed  versions  of  {y„}  at  the  beginning  and 
end  of  the  original  series.  The  Totvar  estimator  makes  use  of  certain  of  these  surrogate  values 
in  order  to  come  up  with  a  new  estimator  of  the  Allan  variance  that  has  better  mean-squared 
error  properties  than  the  usual  Allan  variance  estimator  at  the  very  largest  sampling  times 
(Howe  and  Greenhall  [1]).  Here  we  show  that  a  renormalized  version  of  5ie  Totvar  estimator 
can  be  used  to  exactly  decompose  twice  the  sample  variance.  Except  for  the  factor  of  two  (an 
historical  artifact  due  to  the  original  definition  of  the  Allan  variance),  this  decomposition  of 
the  sample  variance  is  very  much  similar  to  the  one  afforded  by  traditional  spectral  analysis 
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estimators,  which  exactly  decompose  the  sample  variance  across  different  Fourier  frequencies. 
By  comparison  our  results  show  that  the  (renormalized)  Totvar  estimator  decomposes  the 
sample  variance  across  dyadic  averaging  times  (i.e.,  averaging  times  of  the  form  2ho,  where  tq 
is  the  sample  period  for  {j/n})-  Our  result  thus  says  that  the  Allan  variance  can  be  regarded 
as  an  example  of  an  analysis  of  variance  technique,  which  is  one  of  the  most  widely  used  data 
analysis  methods  in  modem  statistics. 

The  remainder  of  this  paper  is  organized  as  follows.  In  Section  2  we  recall  that  in  fact 
a  very  early  estimator  of  the  Allan  variance  (the  nonoverlapped  estimator)  exactly  decomposes 
twice  the  sample  variance  for  the  special  cases  when  Ny  is  a  power  of  two.  Because  of  its 
poor  variance  properties,  the  nonoverlapped  estimator  is  very  seldom  used,  so  we  discuss  in 
Section  3  what  is  generally  considered  to  be  the  preferred  estimator,  namely,  the  maximal 
overlap  estimator.  The  usual  formulation  of  this  estimator  does  not  3deld  a  decomposition  of 
twice  the  sample  variance;  however,  if  we  view  this  estimator  as  the  mean-squared  output  of  a 
circular  filtering  operation,  we  can  augment  the  estimator  with  additional  terms  (namely,  ones 
that  make  explicit  use  of  the  circularity  assumption)  and  come  up  with  an  biased  version  of 
the  maximal  overlap  estimator  that  does  yield  a  decomposition  of  twice  the  sample  variance 
for  any  sample  size  Ny.  Because  of  the  potential  mismatch  between  yi  and  this  circularity 
assumption  can  lead  to  serious  biases.  Thus,  in  Section  4  we  consider  using  the  biased  maximal 
overlap  estimator  with  the  series  of  length  2Ny  formed  by  tacking  on  a  reversed  version  of 
{y„}  at  the  end  of  the  original  series.  This  new  estimator  can  be  written  as  a  renormalized 
version  of  the  Totvar  estimator.  In  Section  5  we  summarize  our  results  and  conclude  with  a 
few  comments. 

2  THE  NONOVERLAPPED  ESTIMATOR  OF  THE  ALLAN 
VARIANCE 

For  this  section  only  we  assume  that  the  sample  size  is  a  power  of  two  ;  i.e.,  we  can  write 
Ny  =  2-^  for  a  positive  integer  J.  Given  a  sequence  of  To-average  fractional  frequency  deviates 
{y„  : n  =  l,...,Ny}  with  a  sampling  period  between  adjacent  observations  given  by  tq  also,  let 
us  define  the  mro-average  fractional  frequency  deviate  as 

j  m— 1 

l/n(w)s  — 

I^we  regard  {y„(m)  :  n  =  m,.,.,Ny}  as  a  realization  of  one  portion  of  the  stochastic  process 
{y„(m)  :n  =  0,±l,±2, ...},  the  Allan  variance  for  averaging  time  mro  is  defined  as 

where  we  assume  that  the  stochastic  process  is  such  that  the  expectation  above  in  fact  depends 
on  the  averaging  time  ind^  m,  but  not  on  the  time  index  n  (this  will  be  true  if  the  first 
difference  process  {y„(l) -y„_i(l)}  is  a  stationary  process). 

For  m  =  2^  for  j  =  0, 1,...,  J  -  1,  let  us  form  the  so-called  nonoverlapped  estimator  of 
the  Allan  variance: 


For  example,  if  j  =  0  so  that  m  =  i,  the  above  reduces  to 

=  aT  ^  “  y2k-if  ,  (1) 

k=l 

so  that  each  yn  contributes  to  exactly  one  term  in  the  sum  of  squares  above  (hence  the  origin 
of  the  name  “nonoverlapped  estimator”).  At  the  other  extreme  when  j  =  J  —  l  so  that  m  =  ^, 
we  have  ^ 


The  nonoverlapped  estimator  can  be  interpreted  in  terms  of  an  orthonormal  transform 
of  the  column  vector  y  whose  elements  are  given  by  {;/„}.  For  Ny  =  8,  this  transform  is  given 
by  the  following  8x8  matrix: 
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(for  other  Ny,  the  Ny  x  Ny  matrix  W  is  formulated  in  an  analogous  manner  and  and  is  one 
version  of  the  discrete  Haar  wavelet  transform  -  for  details,  see,  e.g.,  [4]).  Letting  w  =  Wy 
and  letting  {«;«}  denote  the  elements  of  w,  it  follows  that 


Wj  +  Wj  +  4-  wl 

wl  +  wl 

=  4^y,-.(2) 

= 

wl 

=  sf 

Because  W  is  an  orthonormal  transform,  we  must  have  ||w|p  =  ||y|p,  where  llx||  is  the  usual 
Euclidean  norm  of  the  vector  x.  It  follows  that 

n^l  n=:l  j—O 

For  general  Ny  =  2^,  the  corresponding  result  is 

=  5 or,  equivalently,  =  2^=; 

j=0 

i.e.,  summing  the  nonoverlapped  Allan  variance  estimator  over  all  dyadic  averaging  times  less 
than  or  equal  to  ^  yields  exactly  twice  the  sample  variance  (for  additional  details  and  some 
historical  background,  see  Section  III  of  [3]). 
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3  MAXIMAL  OVERLAP  ESTIMATORS  OF  THE  ALLAN 
VARIANCE 


The  nonoverlapped  estimator  of  the  Allan  variance  is  rarely  used  in  practice  because  it  does 
not  take  advantage  of  certain  "information”  regarding  To  see  in  what  sense  this  is  true, 

let  us  consider  the  form  of  the  maximal  overlap  estimator  for  m  =  l: 

1  2 
_ 1)  ^  ^  ‘ 

Note  that,  whereas  each  j/„  appears  exactly  once  in  the  nonoverlapped  estimator  of  Equation 
(1),  the  variables  appear  twice  because  now,  in  addition  to  terms  like  [1/2 -j/i]’, 

[j/4  -  Vi?  and  [ye  -  tisf  that  appear  in  the  maximal  overlap  estimator  also  includes  terms 

like  [ys  -  and  [ys  -  y4f.  For  general  m  the  maximal  overlap  estimator  takes  the  form 


1 

2{Ny  —  2m  +  1) 


Ny 

k=2m 


Even  if  we  were  to  restrict  the  sample  size  Ny  to  be  a  power  of  two  2-^,  it  can  be  argued  that 
in  general 

3=0 

so  the  usual  maximal  overlap  estimator  does  not  constitute  an  analysis  of  twice  the  sample 
variance.  There  is,  however,  an  interesting  way  to  define  a  variation  on  the  maximal  overlap 
estimator  that  in  fact  does  yield  an  exact  analysis  of  variance,  as  the  following  argument  shows. 

We  start  with  two  filters  {^,/}  and  {yo,/}  defined  as  follows: 


.  {h  (h  ^  = 

hoj  =  S  -5.  i  =  1;  and  and  yo.i  s  ^  l,  I  =  1;  and 

I  <  0  or  I  >  2;  0,  I  <  0  or  1  >  2 

(in  the  wavelet  literature,  {hoj)  and  {yo^}  are  two  versions  of  what  are  called  the  Haar  wavelet 
and  scaling  filters  -  for  details,  see,  e.g.,  [4]).  Let  Ho{-)  and  Go(  )  be  the  transfer  functions  for 
{hoj}  and  {yo^}: 


00  00 
Ho{f)  ~  =  ie~*"^sm(7r/)  and  Go{f)  =  V]  =  e~*"^cos(7r/); 

/=!:  — 00  00 

i.e.,  So{-)  is  the  discrete  Fourier  transform  (DFT)  of  {h^j}.  Note  that  we  have 

1Ho(/)1"  +  |5o(/)|®  =  i  for  all /.  (2) 


We  want  to  circularly  filter  {y„}  separately  using  the  filters  {ho,t}  and  {yo,i}.  Formally, 

we  do  so  by  defining  ;  I  =  0 . Ny-i}  and  {yg^  :  I  =  0, . . . ,  IVj,  - 1},  which  are  said  to  be 

{/io^}  and  {yo,/}  periodized  to  length  Ny.  By  definition, 

00 

X/  I  =  0,  ...,lVy  —  1, 

fcss— 00 
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with  a  similar  definition  for  {glj).  U  Ny>  2,  we  have 

(h  i  =  fl  Z  =  0; 

hi,  =  <  ,  /  =  i;  and  and  1=  i;  and 

[o,  2<l<Ny-l;  [o,  2<l<Ny-l; 

if,  however,  iV,,  =  1  so  that  and  each  have  but  a  single  term,  then  hl  ^  =  0  and  =  1, 
It  is  an  easy  exercise  to  show  that  the  DFT  of  can  be  obtained  by  subsampling  the  DFT 
for  i.e., 

E  fe  =  0, . . . ,  iNT,  -  1. 

1=0 

The  finite  sequences  {hg ,}  and  (ffoC^)}  thus  constitute  a  Fourier  transform  pair,  a  relationship 
we  will  express  as 

{hl,:l  =  0,...,Ny~l}^{J:ro(j^):k  =  0,...,Ny-l}. 

Similarly  we  have 

{glj  :l  =  0,...,Ny~l}^  {Go(^) :  k  =  0, . . .  ,Ny  -  1). 


Let  us  now  define 

Wo,n  —  y  '  ^0jt/(n-Z)niod  TV,  and  Vo,n  =  ^  '  5ojy(n~0  mod  TV,)  71  =  1, . . .  i-Ny, 

1=0  1=0 

where  we  define  n  mod  iVy  to  be  n  if  1  <  ti  <  JVy  and  to  be  ti  +  kNy  otherwise,  where  k  is  the 
unique  nonzero  integer  such  that  1  <  tH-  kNy  <  Ny  (thus  -1  mod  Ny  =  Ny-l;  0  mod  Ny  =  Ny; 
1  mod  iVy  =  l;  . . . ;  Ny  mod  Ny  =  Ny',  Ny  +  1  mod  Ny  =  1;  etc.).  By  construction  we  have 

2 

jrrrr  12^^  =  (3) 

i.e.,  we  have  expressed  the  maximal  overlap  estimator  of  the  Allan  variance  for  tti  =  1  in  terms 
of  a  sum  of  squares  of  the  output  from  circular  filtering  {j/n}  with  {ftg,}. 

An  important  point  to  note  is  that  &? . fl')  does  not  involve  the  entire  output  from 

the  filter:  it  is  missing  wqj  oc  -  g,,  which  is  the  only  term  that  explicitly  makes  use  of  the 
circularity  assumption.  Inclusion  of  this  term  is  one  of  the  two  keys  to  defining  a  version  of 
the  maximal  overlap  estimator  that  constitutes  an  analysis  of  variance.  The  other  key  is  to 
recognize  that  &y,„„(2^)  for  j  =  1,2,...  can  be  obtained  by  further  filtering  of  {vo.n}  so  that, 
whereas  contains  information  about  the  veiriations  of  {y„}  at  to  averaging  times,  the  series 
{vo,n}  contains  information  about  variations  of  {y„}  at  all  dyadic  averaging  times  higher  than  tq 
(i.e.,  2to,  4to,  etc.).  Accordingly,  let  wq  be  an  Ny  dimensional  vector  whose  elements  are 
and  define  vo  to  contain  {vo,n}-  Letting  {3?^}  be  the  DFT  of  {jt„},  we  have  (firom  a  standard 
theorem  in  filtering  theory) 

{wo,n}  ^  {Ho(^^)yk}  and  ^  {5o(^)3^t}. 
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Parseval’s  theorem  tells  us  that 


K-l 


ll*»ll’  =  ^  E  a"<l  llvop  |Go(i)l’WI’, 

which  in  turn  yields 


fc=0 


Jb=0 


N,-l 


N~.l|2  ^  IIC;-I|2 


-  ^  E  + l5"(i)P)  =  4  E  =  iiyP. 

k=0  Jt=0 

where  we  have  made  use  of  Equation  (2)  and  a  second  application  of  Parseval’s  theorem. 
Let  us  now  define  the  following  estimator  of  the  Allan  variance  for  m  =  l: 

^  n=l  ^ 


We  refer  to  this  estimator  as  the  biased  maximal  overlap  estimator  of  <7®(l)  based  on  It 
differs  from  the  standard  maximal  overlap  estimator  (Equation  (3))  because  of  an  additional 
term  proportional  to  -  yiT-  Although  this  estimator  is  in  general  a  biased  estimator  of  the 
true  Allan  variance,  it  is  in  fact  unbiased  when  {y„}  is  a  white  noise  process.  It  satisfies  the 
analysis  of  variance  condition 

where  fjl^i2)  =  || vo||“  -  2y^ 

■‘''y 

We  can  regard  the  second  piece  of  the  decomposition  fiy,^(2)  as  being  related  to  variations  in 
{y„}  at  dyadic  averaging  times  of  2  and  greater. 

Just  as  {y„}  was  split  into  the  components  {wo.n}  and  we  now  split  {^.n}  into  two 
components,  namely,  {wi,n}  and  The  first  component  {wi,n}  will  be  used  to  construct 

an  estimator  of  <jy(2),  while  the  second  component  is  related  to  variations  in  {!/„}  at  dyadic 
averaging  times  of  4  and  greater.  The  filters  that  accomplish  the  desired  split  are  iVy-periodized 
versions  of  ones  whose  transfer  functions  are  defined  by  Ho{2f)  and  Go{2f)  -  the  impulse 
response  sequence  for  these  filters  can  be  formed  by  taking  the  original  filters  {ho^}  and  {yo,j} 
and  inserting  a  single  zero  after  each  element,  a  procedure  that  is  known  as  upsampling  in 
the  engineering  literature  [4].  For  example,  since  the  /=*  o,  1,2,3  and  4  values  of  the  impulse 
response  sequence  for  HQ{f)  are  given  by  5,  -5,  0,  0  and  0,  the  corresponding  values  for  Ho{2f) 
are  given  by  |,  0,  0  and  0.  We  can  also  obtain  and  by  directly  filtering  {yn}: 


Ny-l  Ny-1 

Wi,n  ~  ^  ^  (n—l) mod Ny  and  ~  V  !  ffl J Vfn-fl mod itf. i  n  = 

1=0  1=0 

where  is  the  circular  filter  such  that 

iK)  ^  i^o(wy)^o(.Wy)y>  likewise,  ^  {^oCf  )Go(7^)}. 

Note  that  the  impulse  response  sequence  for  {h^ji}  is  the  circular  convolution  of  the  impulse 
response  sequence  for  ^0(7^)  and  i.e.,  5,  5,  0,  ...convolved  with  i,  0,  -5,  0,  which 
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yields  i,  -i,  0,  ...(as  long  as  Ny  >  4).  This  latter  filter  is  seen  to  be  proportional  to  the 

filtering  operation  commonly  used  in  estimates  of  a^(2). 

Define  wi  as  the  Ny  dimensional  vector  containing  and  let  vi  contain  By 

a  simple  variation  on  the  argument  used  to  establish  ||wo|p  +  ||vo||®  =  llyjp,  we  have 

l|wi||“  +  l|vif  =  ||vo||^ 

Now  define  the  following  estimator  of  the  Allan  variance  for  m  =  2: 

Ny 

52„(2)=5fi;«?,='|-ii5ir. 

n=i 

This  is  the  standard  maximal  overlap  estimator  with  three  additional  terms  -  these  are  propor¬ 
tional  to  (yi  +  yNy  ~  yNy-2  -  VNy-if,  (vi  +y2-  VNy-i  -  VNy)^  aud  (ya  •+•  ya  -  yi  -  In  general  this 
estimator  is  a  biased  estimator  of  <r“(2).  We  have  the  analysis  of  variance  condition 

=  2al,  where  ^®_(4)  =  -  V- 


We  can  now  state  the  result  for  general  J,  a  proof  of  which  follows  from  an  easy 
inductive  argument.  We  define 


Ny-I  Ny-1 

^  ^  ^jjy(n-l)iticid  Ny  and  Vj^n  =  mod  JV,  i  n  =  1, 

1=0  1=0 

where 

{iy  —  {ff,(^)G,(JS^)Go(3^) . . .  Go(A)) 
and 

^ '  {<5o(^)Go(^)Go(2^)  ■  •  •  Go(j|^)}. 

An  inductive  argument  can  be  used  to  show  that  {h^i)  is  the  usual  filter  involved  in  estimating 
«7*(2^).  Letting  wy  and  vy  be  JVy  dimensional  vectors  containing  {%„}  and  define 


K 


jy 

n=l 


which  is  the  biased  maximal  overlap  estimator  of  <t®(2>)  based  upon  {y„}  -  it  dilfers  from  the 
standard  maximal  overlap  estimator  due  to  2^+^  —  1  additional  terms  involving  explicit  circular 
use  of  {y„}.  For  any  J,  the  biased  maximal  overlap  estimators  satisfy  the  analysis  of  variance 
condition  ^ 

E  +^....(2"’)  =  2SJ  with  =  A||v^f  _  2f, 

j=o 

where  the  term  ng — represents  variations  in  {y„}  at  dyadic  averaging  times  of  2-^+^  and 
greater. 
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4  ESTIMATION  OF  THE  ALLAN  VARIANCE  USING  A 
CIRCULARIZED  SERIES 


For  most  models  for  {y„}  of  interest  for  actual  frequency  standards,  a  circularity  assumption 
can  yield  an  unacceptably  large  bias  in  the  estimator  ^1^(2^')  due  to  the  fact  that  pi  and 
can  be  quite  different.  To  solve  this  problem,  we  construct  a  series  {j/*}  of  length  2Ny 

*  =  1  <  n  <  and 

“  1  J/2JV,+i-n,  Nf  +  l<n<  2Ny. 

For  example,  if  Tfy  =  3  so  that  only  yi,  yj  and  are  observed,  the  values  of  yl,...,y|  are  given, 
respectively,  by  pi,p2>ps,pa,p2,yi‘  Note  that,  by  construction,  the  sample  mean  and  variance  of 
{pn}  and  {y*}  are  identical.  We  now  apply  the  estimation  procedure  of  Section  3  to  {p^}  to 
obtain  the  following  estimator  of  al(2^): 


where  wj  is  a  vector  of  length  2Ny  formed  by  circularly  filtering  {y*}  with  the  circular  filter  of 
length  2Ny  whose  DFT  is  given  by 

■^o(^)5o(|j^)Go(^)  •  •  •  Go(3^),  =  0, . . . ,  2Ny  -  1. 

We  refer  to  cr®,  as  the  biased  maximal  overlap  estimator  of  trj(2^)  based  on  the  circularized 
series  {yj}  (note,  however,  that  this  estimator  is  in  fact  unbiased  for  the  special  case  where 
{y„}  is  a  white  noise  process).  This  biased  estimator  satisfies  the  following  analysis  of  variance 
condition  for  all  J  and  all  sample  sizes  Nyt 

+  "here 

Finally  we  note  that  there  is  a  very  simple  relationship  between  of*.  ^(2^')  and  Totvar  estimator 
(Greenhall,  1997,  private  communication): 

Totvar(2^i\ry,To)  = 

where  Totvar  is  defined  as  in  Equation  (4)  of  Howe  and  Greenhall  [1]. 


5  SUMMARY  AND  COMMENTS 

We  have  developed  a  relationship  between  the  sample  variance  a*  and  the  Totvar  estimator 
Totvar(m,lV'y,To)  of  the  Allan  variance  a*(Tn),  where  m  sets  the  averaging  time  mro,  Ny  is  the 
number  of  To-average  fractional  frequency  deviates  {y„},  and  tq  is  the  basic  sampling  and 
averaging  time  of  the  observed  deviates.  For  any  sample  size  Ny  and  any  positive  integer  J, 
we  have  demonstrated  that 

H  Totvar(2*,  Ny,  To)  +  ^^.^(2-^+^)  =  2^*, 
y  i=a 
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where  can  be  interpreted  in  terms  of  variations  in  {j/n}  at  all  dyadic  averaging  times 

greater  than  2^.  We  have  also  shown  that  the  Totvar  estimator  is  related  to  a  biased  maximal 
overlap  estimator  of  the  Allan  variance  that  is  based  upon  which  is  a  sequence  of  length 
2Ny  formed  by  tacking  onto  {j/„}  a  reversed  version  of  itself. 

In  closing,  we  make  the  following  comments  about  our  results,  some  of  which  will  be 
expanded  upon  in  future  research. 

•  It  can  be  shown  that,  if  {j/n}  is  a  portion  of  a  realization  of  a  stationary  or  nonstationary 
process  {3^}  for  which  the  Allan  variance  is  well-defined,  then  we  have 

j=Q 

where  o-®  is  the  process  variance  of  {i;,}  (this  is  taken  to  be  infinite  if  {K}  is  nonstationary). 
The  estimator  is  the  first  “modem”  estimator  of  the  Allan  variance  to  mimic 

this  important  property. 

•  Because  higher  order  Daubechies  wavelet  filters  also  satisfy  Equation  (2),  the  above 
development  extends  trivially  to  higher  order  wavelet  variances  (the  Allan  variance  is 
essentially  twice  the  Haar  wavelet  variance).  These  higher  order  wavelet  variances  are 
suitable  substitutes  for  some  of  the  variations  on  the  Allan  variance  that  have  been 
proposed  md  studied  in  the  literature  (an  example  is  the  modified  Allan  variance).  For 
details,  see  [2]. 

•  In  addition  to  plotting  versus  2^tq  on  a  log-log  scale,  we  suggest  that  ng. _ (2-^+^) 

be  plotted  (with  a  separate  ^mbol)  versus  2'’'^^to  -  this  will  indicate  how  much  of  has 
not  been  accounted  for  by  estimates  of  the  Allan  variance. 

•  In  theory  J  can  be  made  as  large  as  desired,  but  there  will  be  serious  biases  in  erg. _ (2-^) 

for  any  J  such  that  2-^  >  Ny.  Because  of  its  close  relationship  to  Totvar,  the  results  of 
Howe  and  Greenhall  [1]  indicate  that  ag.,^(2^)  outperforms  traditional  estimators  of  the 
Allan  variance  for  averaging  times  close  to 
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